Libraries
These are the libraries to load
pkg <- c('pivottabler', 'tidyverse','tibbletime','anomalize','timetk')
#install.packages(pkg)
library(pivottabler)
library(lubridate)
library(dplyr)
library(ggplot2)
library(tidyverse)
library(tibbletime)
library(anomalize)
library(timetk)
library(forecast)
Read the revenue and AdSpend
We check the date type of the ‘date’ column and change the type to
DATE
The CSV data is avaialble in the github repository
setwd("/Users/gerardogandara/Documents/recast/")
## Read file as CSV
df_spend <- read.csv('https://raw.githubusercontent.com/ggandara13/myrepository/master/acme_spend.csv')
df_revenue <- read.csv('https://raw.githubusercontent.com/ggandara13/myrepository/master/acme_revenue.csv')
# Convert char format to DATE format
df_revenue$date <- as.Date(df_revenue$date, format = "%m/%d/%y")
df_spend$date <- as.Date(df_spend$date, format = "%m/%d/%y")
# Check the format and content of each dataframe
str(df_revenue)
'data.frame': 1035 obs. of 5 variables:
$ X : int 1 2 3 4 5 6 7 8 9 10 ...
$ date : Date, format: "2020-01-01" "2020-01-02" "2020-01-03" ...
$ revenue_dtc : num 16000 19306 22052 23177 26169 ...
$ revenue_amazon : num 50000 60332 68914 72428 81778 ...
$ revenue_walmart: num 34000 41026 46861 49251 55609 ...
str(df_spend)
'data.frame': 10350 obs. of 4 variables:
$ X : int 1 2 3 4 5 6 7 8 9 10 ...
$ date : Date, format: "2020-01-01" "2020-01-01" "2020-01-01" ...
$ channel: chr "facebook_prospecting" "facebook_retargeting" "google_branded_search" "google_nonbranded_search" ...
$ spend : num 4154 2679 474 3817 7421 ...
Check there are not NULL values
paste("Total of NULL in Spend:", sum(is.na(df_spend)))
[1] "Total of NULL in Spend: 0"
paste("\nTotal of NULL in Revenue:", sum(is.na(df_revenue)))
[1] "\nTotal of NULL in Revenue: 0"
paste("\n\nSummary\n")
[1] "\n\nSummary\n"
# you can check with summary as well and see the columns
summary(df_spend)
X date channel spend Year
Min. : 1 Min. :2020-01-01 Length:10350 Min. : 0 Min. :2020
1st Qu.: 2588 1st Qu.:2020-09-15 Class :character 1st Qu.: 1278 1st Qu.:2020
Median : 5176 Median :2021-06-01 Mode :character Median : 5268 Median :2021
Mean : 5176 Mean :2021-06-01 Mean : 7740 Mean :2021
3rd Qu.: 7763 3rd Qu.:2022-02-15 3rd Qu.:11649 3rd Qu.:2022
Max. :10350 Max. :2022-10-31 Max. :36401 Max. :2022
summary(df_revenue)
X date revenue_dtc revenue_amazon revenue_walmart
Min. : 1.0 Min. :2020-01-01 Min. : 0 Min. : 0 Min. : 0
1st Qu.: 259.5 1st Qu.:2020-09-15 1st Qu.: 41643 1st Qu.:130216 1st Qu.: 88547
Median : 518.0 Median :2021-06-01 Median : 72051 Median :225161 Median :153109
Mean : 518.0 Mean :2021-06-01 Mean : 72741 Mean :227062 Mean :154488
3rd Qu.: 776.5 3rd Qu.:2022-02-14 3rd Qu.: 97340 3rd Qu.:303352 3rd Qu.:206279
Max. :1035.0 Max. :2022-10-31 Max. :193005 Max. :415952 Max. :282848
sum(is.na(df_spend))/nrow(df_spend)*100
[1] 0
sum(is.na(df_revenue))/nrow(df_revenue)*100
[1] 0
Since we have 1,035 rows, we can remove because that represents less
than 1% or just replace with zero
Now, let’s replace with zero since it is a time series
df_revenue[is.na(df_revenue)] <- 0
df_spend[is.na(df_spend)] <- 0
cat("Total of NULL in Spend:", sum(is.na(df_spend)))
Total of NULL in Spend: 0
cat("\nTotal of NULL in Revenue:", sum(is.na(df_revenue)))
Total of NULL in Revenue: 0
1- Which channel had the largest increase in spend in 2022 compared
to the same date range in 2021?
Using lubridate to filter the year of the date, also use the
pivottabler library to group/aggregate it
Answer: online_video
# print the first value of the dataframe
paste("The channel with highest LIFT is:", row.names(df_summary_final)[1] )
[1] "The channel with highest LIFT is: online_video"
2- In terms of total revenue, are there any anomalous days?
Group by day to see the trend and plot
df_revenue_total <- df_revenue %>%
mutate(total_revenue = ( df_revenue$revenue_dtc + df_revenue$revenue_amazon + df_revenue$revenue_walmart ))
# Group by sum using R Base aggregate()
agg_df <- aggregate(df_revenue_total$total_revenue, by=list(df_revenue_total$date), FUN=sum)
colnames(agg_df) <- c('date','revenue')
#Plot sales over 36 months
ggplot(agg_df, aes(x=date, y = revenue)) + geom_line()

paste("Visually we can see some outliers")
[1] "Visually we can see some outliers"
Now, we can use the anomalties library to identify the values
Use time_decompose() to decompose a time series prior to performing
anomaly detection with anomalize(). Typically, anomalize() is performed
on the “remainder” of the time series decomposition.
The return has three columns: “remainder_l1” (lower limit for
anomalies), “remainder_l2” (upper limit for anomalies), and “anomaly”
(Yes/No).

We can print the detail of the anomalies, for example the last 4
tail(df_anomalized %>%
filter(anomaly == 'Yes'),4)
NA
You can also use the timetk package for dynamic plot
agg_df %>% timetk::plot_anomaly_diagnostics(date,revenue, .facet_ncol = 2)
frequency = 7 observations per 1 week
trend = 92 observations per 3 months
3- In which month of the year does Acme tend to make the most
revenue?
We can compute month column before we aggregate and sort to get the
popular month
Most profitale month in a year is the month number: 3
# Calculate the month column
monthfunction<-function(dataframe, datecolumn) {
month(dataframe[,datecolumn])
}
agg_df$Month <- monthfunction(agg_df, "date")
# group by month
by_month <- aggregate(agg_df$revenue, by = list(agg_df$Month), FUN = sum)
colnames(by_month) <- c('month','revenue')
by_month <- by_month[order(by_month$revenue, decreasing = TRUE), ]
# print the first value of the dataframe
print(by_month)
paste("Most profitale month in a year is the month number:", row.names(by_month)[1] )
[1] "Most profitale month in a year is the month number: 3"
4- Does Acme’s marketing spend tend to follow a similar pattern to
revenue?
We need to merge both dataframes to have the TS of Revenue using
AdSped as regressor Convert the dataframe to TS then plot, then plot the
results
Visually we can confirm they have a similar pattern.
#select the date and total_revenue of Acme
df_only_revenue <- select(df_revenue_total, date, total_revenue)
#aggregate spend of all media
df_spend_total <- aggregate(df_spend$spend, by=list(df_spend$date), FUN=sum)
colnames(df_spend_total) <- c('date','spend')
df_rev_spend <- merge(df_only_revenue, df_spend_total, by = "date")
#Convert dataframe to time series object using the ts() function
acme_ts <- ts(data = df_rev_spend[,c(2,3)])
# Time plot of both variables
autoplot(acme_ts, facets = TRUE)

5- Based on what you see here, could you make the case that
marketing spend is causally related to revenue? Why or why not?
Visually we can see the effect has high correlation, and when we
calculate the R2 is 0.91 with p=value basically zero. So, it is
statiscally correlated.

Is the relation linear? Yes, form the plot above, the relationship is
linear.
now we jsut need to verify that the data from each of the 2 variables
(revenue, spend) follow a normal distribution.
df_rev_spend
# Shapiro-Wilk normality test for revenue
shapiro.test(df_rev_spend$total_revenue) # => p-value < 2.2e-16
Shapiro-Wilk normality test
data: df_rev_spend$total_revenue
W = 0.93373, p-value < 2.2e-16
# Shapiro-Wilk normality test for spend
shapiro.test(df_rev_spend$spend) # => p-value < 2.2e-16
Shapiro-Wilk normality test
data: df_rev_spend$spend
W = 0.95587, p-value < 2.2e-16
# revenue
ggqqplot(df_rev_spend$total_revenue, ylab = "revenue")

# spend
ggqqplot(df_rev_spend$spend, ylab = "spend")

NA
NA
Since the p-value is practically zero, we can reject the null
hypothesis that it is normal distributed both variables
so we can use another test to check asociation Kendall rank
correlation test - Kendall’s tau
This is used to estimate a rank-based measure of association. This
test may be used if the data do not necessarily come from a bivariate
normal distribution.
The correlation coefficient between revenue and spend are 0.7209243
and the p-value < 2.2e-16. Exist correlation
but we know it is not causation
res2
Kendall's rank correlation tau
data: df_rev_spend$total_revenue and df_rev_spend$spend
z = 34.731, p-value < 2.2e-16
alternative hypothesis: true tau is not equal to 0
sample estimates:
tau
0.7209243
We know that they are highly correlated. We can use time series to
see the magnitude of the effect of marketing using it as regressor
Visually in the previous plot of revenue, we can see it is
stationary, and there is seasonality. We can use auto.arima to calculate
the regressor factor
This is a very basic analysis, but it helps to understand in overall
how the marketing efforts are creating an effect in the revenue.
1.489071 is the regressor factor
Interpretation of sales increase: For every $1,000 (the unit of ad
spend is in thousands USD) increase in advertising spend, the unit sales
increase by 1.4891 - It is a positive effect, so ACME probably is using
RECAST and the marketing campaign is performaing well
sales_increase
xreg
1.489071
LS0tCnRpdGxlOiAiQXNzaWdubWVudC1SRUNBU1QgTW9kZXJuIE1hcmtldGluZyBNaXggTW9kZWxpbmcgIgpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDoKICAgIGRmX3ByaW50OiBwYWdlZAotLS0KCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIwojICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICMKIyBQdXJwb3NlOiAgICAgICBSZXZlbnVlIHZzIEFkU3BlbmQgZWZmZWN0ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjCiMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIwojIEF1dGhvcjogICAgICAgIEdlcmFyZG8gR2FuZGFyYSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICMKIyBDb250YWN0OiAgICAgICBnZXJhcmRvLmdhbmRhcmFAZ21haWwuY29tICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjCiMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIwojIENsaWVudDogICAgICAgIGFuZHlAZ2V0cmVjYXN0LmNvbSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICMKIyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjCiMgQ29kZSBjcmVhdGVkOiAgMjAyMy0xMC0xMSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIwojIExhc3QgdXBkYXRlZDogIDIwMjMtMTAtMTEgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICMKIyBTb3VyY2U6ICAgICAgICBodHRwczovL2dpdGh1Yi5jb20vZ2dhbmRhcmExMy9teXJlcG9zaXRvcnkgICAgICAgICAgICAgICAjCiMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIwojIENvbW1lbnQ6ICAgICAgIGh0dHBzOi8vZ2V0cmVjYXN0LmNvbS9tb2Rlcm4tbWVkaWEtbWl4LW1vZGVsaW5nLy4gICAgICAgICMKIyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIwoKCiMgTGlicmFyaWVzClRoZXNlIGFyZSB0aGUgbGlicmFyaWVzIHRvIGxvYWQKCgpgYGB7cn0KcGtnIDwtIGMoJ3Bpdm90dGFibGVyJywgJ3RpZHl2ZXJzZScsJ3RpYmJsZXRpbWUnLCdhbm9tYWxpemUnLCd0aW1ldGsnKQojaW5zdGFsbC5wYWNrYWdlcyhwa2cpCgpsaWJyYXJ5KHBpdm90dGFibGVyKQpsaWJyYXJ5KGx1YnJpZGF0ZSkKbGlicmFyeShkcGx5cikKbGlicmFyeShnZ3Bsb3QyKQpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeSh0aWJibGV0aW1lKQpsaWJyYXJ5KGFub21hbGl6ZSkKbGlicmFyeSh0aW1ldGspCmxpYnJhcnkoZm9yZWNhc3QpCgpgYGAKCgoKUmVhZCB0aGUgcmV2ZW51ZSBhbmQgQWRTcGVuZAoKV2UgY2hlY2sgdGhlIGRhdGUgdHlwZSBvZiB0aGUgJ2RhdGUnIGNvbHVtbiBhbmQgY2hhbmdlIHRoZSB0eXBlIHRvIERBVEUKClRoZSBDU1YgZGF0YSBpcyBhdmFpYWxibGUgaW4gdGhlIGdpdGh1YiByZXBvc2l0b3J5CgoKYGBge3J9CgpzZXR3ZCgiL1VzZXJzL2dlcmFyZG9nYW5kYXJhL0RvY3VtZW50cy9yZWNhc3QvIikKCiMjIFJlYWQgZmlsZSBhcyBDU1YKZGZfc3BlbmQgPC0gcmVhZC5jc3YoJ2h0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9nZ2FuZGFyYTEzL215cmVwb3NpdG9yeS9tYXN0ZXIvYWNtZV9zcGVuZC5jc3YnKQpkZl9yZXZlbnVlIDwtIHJlYWQuY3N2KCdodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vZ2dhbmRhcmExMy9teXJlcG9zaXRvcnkvbWFzdGVyL2FjbWVfcmV2ZW51ZS5jc3YnKQoKIyBDb252ZXJ0IGNoYXIgZm9ybWF0IHRvIERBVEUgZm9ybWF0CmRmX3JldmVudWUkZGF0ZSA8LSBhcy5EYXRlKGRmX3JldmVudWUkZGF0ZSwgZm9ybWF0ID0gICIlbS8lZC8leSIpCmRmX3NwZW5kJGRhdGUgPC0gYXMuRGF0ZShkZl9zcGVuZCRkYXRlLCBmb3JtYXQgPSAgIiVtLyVkLyV5IikKCiMgQ2hlY2sgdGhlIGZvcm1hdCBhbmQgY29udGVudCBvZiBlYWNoIGRhdGFmcmFtZQpzdHIoZGZfcmV2ZW51ZSkKc3RyKGRmX3NwZW5kKQoKYGBgCgoKQ2hlY2sgdGhlcmUgYXJlIG5vdCBOVUxMIHZhbHVlcyAKCmBgYHtyfQoKcGFzdGUoIlRvdGFsIG9mIE5VTEwgaW4gU3BlbmQ6Iiwgc3VtKGlzLm5hKGRmX3NwZW5kKSkpCgpwYXN0ZSgiXG5Ub3RhbCBvZiBOVUxMIGluIFJldmVudWU6Iiwgc3VtKGlzLm5hKGRmX3JldmVudWUpKSkKCnBhc3RlKCJcblxuU3VtbWFyeVxuIikKIyB5b3UgY2FuIGNoZWNrIHdpdGggc3VtbWFyeSBhcyB3ZWxsIGFuZCBzZWUgdGhlIGNvbHVtbnMKc3VtbWFyeShkZl9zcGVuZCkKc3VtbWFyeShkZl9yZXZlbnVlKQoKc3VtKGlzLm5hKGRmX3NwZW5kKSkvbnJvdyhkZl9zcGVuZCkqMTAwCnN1bShpcy5uYShkZl9yZXZlbnVlKSkvbnJvdyhkZl9yZXZlbnVlKSoxMDAKCgpgYGAKU2luY2Ugd2UgaGF2ZSAxLDAzNSByb3dzLCB3ZSBjYW4gcmVtb3ZlIGJlY2F1c2UgdGhhdCByZXByZXNlbnRzIGxlc3MgdGhhbiAxJSBvciBqdXN0IHJlcGxhY2Ugd2l0aCB6ZXJvCgpOb3csIGxldCdzIHJlcGxhY2Ugd2l0aCB6ZXJvIHNpbmNlIGl0IGlzIGEgdGltZSBzZXJpZXMKYGBge3J9CgpkZl9yZXZlbnVlW2lzLm5hKGRmX3JldmVudWUpXSA8LSAwCmRmX3NwZW5kW2lzLm5hKGRmX3NwZW5kKV0gPC0gMAoKCmNhdCgiVG90YWwgb2YgTlVMTCBpbiBTcGVuZDoiLCBzdW0oaXMubmEoZGZfc3BlbmQpKSkKY2F0KCJcblRvdGFsIG9mIE5VTEwgaW4gUmV2ZW51ZToiLCBzdW0oaXMubmEoZGZfcmV2ZW51ZSkpKQoKCmBgYAoKCgoKIyMgMS0gV2hpY2ggY2hhbm5lbCBoYWQgdGhlIGxhcmdlc3QgaW5jcmVhc2UgaW4gc3BlbmQgaW4gMjAyMiBjb21wYXJlZCB0byB0aGUgc2FtZSBkYXRlIHJhbmdlIGluIDIwMjE/CgpVc2luZyBsdWJyaWRhdGUgdG8gZmlsdGVyIHRoZSB5ZWFyIG9mIHRoZSBkYXRlLCBhbHNvIHVzZSB0aGUgcGl2b3R0YWJsZXIgbGlicmFyeSB0byBncm91cC9hZ2dyZWdhdGUgaXQKCkFuc3dlcjogb25saW5lX3ZpZGVvCgoKYGBge3J9CnllYXJmdW5jdGlvbjwtZnVuY3Rpb24oZGF0YWZyYW1lLCBkYXRlY29sdW1uKSB7CiAgICB5ZWFyKGRhdGFmcmFtZVssZGF0ZWNvbHVtbl0pCn0KZGZfc3BlbmQkWWVhciA8LSB5ZWFyZnVuY3Rpb24oZGZfc3BlbmQsICJkYXRlIikKCgpkZiA8LSBkZl9zcGVuZCAlPiUgZ3JvdXBfYnkoY2hhbm5lbCkgJT4lIAogIGZpbHRlcihsdWJyaWRhdGU6OnllYXIoZGF0ZSkgJWluJSBjKDIwMjEsIDIwMjIpICkKZGYgPC0gYXMuZGF0YS5mcmFtZShkZikKCgojIGluaXRpYWxpY2UgdGhlIFBJVk9UCnB0IDwtIFBpdm90VGFibGUkbmV3KCkKIyBhZGQgdGhlIGRmIHdpdGggMiB5ZWFycwpwdCRhZGREYXRhKGRmKQpwdCRhZGRDb2x1bW5EYXRhR3JvdXBzKCJZZWFyIikKcHQkYWRkUm93RGF0YUdyb3VwcygiY2hhbm5lbCIpCnB0JGRlZmluZUNhbGN1bGF0aW9uKGNhbGN1bGF0aW9uTmFtZT0iVG90YWxTcGVuZCIsIHN1bW1hcmlzZUV4cHJlc3Npb249InN1bShzcGVuZCkiKQpwdCRldmFsdWF0ZVBpdm90KCkKCiNjb3ZuZXJ0IHRvIGRhdGFmcmFtZQpkZjEgPC0gcHQkYXNEYXRhRnJhbWUoKQpkZl9zdW1tYXJ5X2ZpbmFsIDwtIGRmMSAlPiUKICBtdXRhdGUoaW5kZXhfbGlmdCA9ICggKGRmMVssMl0vIGRmMVssMV0pIC0gMSApICoxMDAgICkKCmRmX3N1bW1hcnlfZmluYWw8LSBkZl9zdW1tYXJ5X2ZpbmFsW29yZGVyKGRmX3N1bW1hcnlfZmluYWwkaW5kZXhfbGlmdCwgZGVjcmVhc2luZyA9IFRSVUUpLCBdCgoKIyBwcmludCB0aGUgZmlyc3QgdmFsdWUgb2YgdGhlIGRhdGFmcmFtZQpwYXN0ZSgiVGhlIGNoYW5uZWwgd2l0aCBoaWdoZXN0IExJRlQgaXM6Iiwgcm93Lm5hbWVzKGRmX3N1bW1hcnlfZmluYWwpWzFdICkKCgpgYGAKCgoKCiMjIDItIEluIHRlcm1zIG9mIHRvdGFsIHJldmVudWUsIGFyZSB0aGVyZSBhbnkgYW5vbWFsb3VzIGRheXM/CgpHcm91cCBieSBkYXkgdG8gc2VlIHRoZSB0cmVuZCBhbmQgcGxvdAoKCmBgYHtyfQoKZGZfcmV2ZW51ZV90b3RhbCA8LSBkZl9yZXZlbnVlICU+JQogIG11dGF0ZSh0b3RhbF9yZXZlbnVlID0gKCBkZl9yZXZlbnVlJHJldmVudWVfZHRjICsgZGZfcmV2ZW51ZSRyZXZlbnVlX2FtYXpvbiArIGRmX3JldmVudWUkcmV2ZW51ZV93YWxtYXJ0ICApKQoKIyBHcm91cCBieSBzdW0gdXNpbmcgUiBCYXNlIGFnZ3JlZ2F0ZSgpCmFnZ19kZiA8LSBhZ2dyZWdhdGUoZGZfcmV2ZW51ZV90b3RhbCR0b3RhbF9yZXZlbnVlLCBieT1saXN0KGRmX3JldmVudWVfdG90YWwkZGF0ZSksIEZVTj1zdW0pCmNvbG5hbWVzKGFnZ19kZikgPC0gYygnZGF0ZScsJ3JldmVudWUnKSAKCgojUGxvdCBzYWxlcyBvdmVyIDM2IG1vbnRocwpnZ3Bsb3QoYWdnX2RmLCBhZXMoeD1kYXRlLCB5ID0gcmV2ZW51ZSkpICsgZ2VvbV9saW5lKCkKCnBhc3RlKCJWaXN1YWxseSB3ZSBjYW4gc2VlIHNvbWUgb3V0bGllcnMiKQpgYGAKCk5vdywgd2UgY2FuIHVzZSB0aGUgYW5vbWFsdGllcyBsaWJyYXJ5IHRvIGlkZW50aWZ5IHRoZSB2YWx1ZXMKClVzZSB0aW1lX2RlY29tcG9zZSgpIHRvIGRlY29tcG9zZSBhIHRpbWUgc2VyaWVzIHByaW9yIHRvIHBlcmZvcm1pbmcgYW5vbWFseSBkZXRlY3Rpb24gd2l0aCBhbm9tYWxpemUoKS4gVHlwaWNhbGx5LCBhbm9tYWxpemUoKSBpcyBwZXJmb3JtZWQgb24gdGhlICJyZW1haW5kZXIiIG9mIHRoZSB0aW1lIHNlcmllcyBkZWNvbXBvc2l0aW9uLgoKVGhlIHJldHVybiBoYXMgdGhyZWUgY29sdW1uczogInJlbWFpbmRlcl9sMSIgKGxvd2VyIGxpbWl0IGZvciBhbm9tYWxpZXMpLCAicmVtYWluZGVyX2wyIiAodXBwZXIgbGltaXQgZm9yIGFub21hbGllcyksIGFuZCAiYW5vbWFseSIgKFllcy9ObykuCgoKCmBgYHtyfQojIENvbnZlcnQgZGYgdG8gYSB0aWJibGUKZGYgPC0gYXNfdGliYmxlKGFnZ19kZikKY2xhc3MoZGYpCgpkZl9hbm9tYWxpemVkIDwtIGRmICU+JQogICAgdGltZV9kZWNvbXBvc2UocmV2ZW51ZSwgbWV0aG9kID0gInN0bCIsIG1lcmdlID0gVFJVRSkgJT4lCiAgICBhbm9tYWxpemUocmVtYWluZGVyLCBtZXRob2QgPSAiaXFyIiwgYWxwaGEgPSAwLjA1LCBtYXhfYW5vbXMgPSAwLjIpICU+JQogICAgdGltZV9yZWNvbXBvc2UoKQoKI1dlIGNhbiB0aGVuIHZpc3VhbGl6ZSB0aGUgYW5vbWFsaWVzIHVzaW5nIHRoZSBwbG90X2Fub21hbGllcygpIGZ1bmN0aW9uLiAKZGZfYW5vbWFsaXplZCAlPiUgcGxvdF9hbm9tYWxpZXMobmNvbCA9IDEsIGFscGhhX2RvdHMgPSAwLjc1KQoKYGBgCgpXZSBjYW4gcHJpbnQgdGhlIGRldGFpbCBvZiB0aGUgYW5vbWFsaWVzLCBmb3IgZXhhbXBsZSB0aGUgbGFzdCA0CgoKYGBge3J9CnRhaWwoZGZfYW5vbWFsaXplZCAlPiUKICBmaWx0ZXIoYW5vbWFseSA9PSAnWWVzJyksNCkKYGBgCgpZb3UgY2FuIGFsc28gdXNlIHRoZSB0aW1ldGsgcGFja2FnZSBmb3IgZHluYW1pYyBwbG90CgpgYGB7cn0KYWdnX2RmICU+JSB0aW1ldGs6OnBsb3RfYW5vbWFseV9kaWFnbm9zdGljcyhkYXRlLHJldmVudWUsIC5mYWNldF9uY29sID0gMikKCmBgYAoKCiMjIDMtIEluIHdoaWNoIG1vbnRoIG9mIHRoZSB5ZWFyIGRvZXMgQWNtZSB0ZW5kIHRvIG1ha2UgdGhlIG1vc3QgcmV2ZW51ZT8KCldlIGNhbiBjb21wdXRlIG1vbnRoIGNvbHVtbiBiZWZvcmUgd2UgYWdncmVnYXRlIGFuZCBzb3J0IHRvIGdldCB0aGUgcG9wdWxhciBtb250aAoKTW9zdCBwcm9maXRhbGUgbW9udGggaW4gYSB5ZWFyIGlzIHRoZSBtb250aCBudW1iZXI6IDMKCmBgYHtyfQoKIyBDYWxjdWxhdGUgdGhlIG1vbnRoIGNvbHVtbgptb250aGZ1bmN0aW9uPC1mdW5jdGlvbihkYXRhZnJhbWUsIGRhdGVjb2x1bW4pIHsKICAgIG1vbnRoKGRhdGFmcmFtZVssZGF0ZWNvbHVtbl0pCn0KYWdnX2RmJE1vbnRoIDwtIG1vbnRoZnVuY3Rpb24oYWdnX2RmLCAiZGF0ZSIpCgojIGdyb3VwIGJ5IG1vbnRoCgpieV9tb250aCA8LSBhZ2dyZWdhdGUoYWdnX2RmJHJldmVudWUsIGJ5ID0gbGlzdChhZ2dfZGYkTW9udGgpLCBGVU4gPSBzdW0pCmNvbG5hbWVzKGJ5X21vbnRoKSA8LSBjKCdtb250aCcsJ3JldmVudWUnKSAKCgoKYnlfbW9udGggPC0gYnlfbW9udGhbb3JkZXIoYnlfbW9udGgkcmV2ZW51ZSwgZGVjcmVhc2luZyA9IFRSVUUpLCBdCgojIHByaW50IHRoZSBmaXJzdCB2YWx1ZSBvZiB0aGUgZGF0YWZyYW1lCnByaW50KGJ5X21vbnRoKQpwYXN0ZSgiTW9zdCBwcm9maXRhbGUgbW9udGggaW4gYSB5ZWFyIGlzIHRoZSBtb250aCBudW1iZXI6Iiwgcm93Lm5hbWVzKGJ5X21vbnRoKVsxXSApCgpgYGAKCgojIyA0LSBEb2VzIEFjbWUncyBtYXJrZXRpbmcgc3BlbmQgdGVuZCB0byBmb2xsb3cgYSBzaW1pbGFyIHBhdHRlcm4gdG8gcmV2ZW51ZT8KCldlIG5lZWQgdG8gbWVyZ2UgYm90aCBkYXRhZnJhbWVzIHRvIGhhdmUgdGhlIFRTIG9mIFJldmVudWUgdXNpbmcgQWRTcGVkIGFzIHJlZ3Jlc3NvcgpDb252ZXJ0IHRoZSBkYXRhZnJhbWUgdG8gVFMgdGhlbiBwbG90LCB0aGVuIHBsb3QgdGhlIHJlc3VsdHMKClZpc3VhbGx5IHdlIGNhbiBjb25maXJtIHRoZXkgaGF2ZSBhIHNpbWlsYXIgcGF0dGVybi4KCmBgYHtyfQoKI3NlbGVjdCB0aGUgZGF0ZSBhbmQgdG90YWxfcmV2ZW51ZSBvZiBBY21lCmRmX29ubHlfcmV2ZW51ZSA8LSBzZWxlY3QoZGZfcmV2ZW51ZV90b3RhbCwgZGF0ZSwgdG90YWxfcmV2ZW51ZSkKCiNhZ2dyZWdhdGUgc3BlbmQgb2YgYWxsIG1lZGlhCmRmX3NwZW5kX3RvdGFsIDwtIGFnZ3JlZ2F0ZShkZl9zcGVuZCRzcGVuZCwgYnk9bGlzdChkZl9zcGVuZCRkYXRlKSwgRlVOPXN1bSkKY29sbmFtZXMoZGZfc3BlbmRfdG90YWwpIDwtIGMoJ2RhdGUnLCdzcGVuZCcpIAoKZGZfcmV2X3NwZW5kIDwtIG1lcmdlKGRmX29ubHlfcmV2ZW51ZSwgZGZfc3BlbmRfdG90YWwsIGJ5ID0gImRhdGUiKQoKCiNDb252ZXJ0IGRhdGFmcmFtZSB0byB0aW1lIHNlcmllcyBvYmplY3QgdXNpbmcgdGhlIHRzKCkgZnVuY3Rpb24KYWNtZV90cyA8LSB0cyhkYXRhID0gZGZfcmV2X3NwZW5kWyxjKDIsMyldKQoKCiMgVGltZSBwbG90IG9mIGJvdGggdmFyaWFibGVzCmF1dG9wbG90KGFjbWVfdHMsIGZhY2V0cyA9IFRSVUUpCgpgYGAKCgoKCgoKIyMgNS0gQmFzZWQgb24gd2hhdCB5b3Ugc2VlIGhlcmUsIGNvdWxkIHlvdSBtYWtlIHRoZSBjYXNlIHRoYXQgbWFya2V0aW5nIHNwZW5kIGlzIGNhdXNhbGx5IHJlbGF0ZWQgdG8gcmV2ZW51ZT8gV2h5IG9yIHdoeSBub3Q/CgpWaXN1YWxseSB3ZSBjYW4gc2VlIHRoZSBlZmZlY3QgaGFzIGhpZ2ggY29ycmVsYXRpb24sIGFuZCB3aGVuIHdlIGNhbGN1bGF0ZSB0aGUgUjIgaXMgMC45MSB3aXRoIHA9dmFsdWUgYmFzaWNhbGx5IHplcm8uIFNvLCBpdCBpcyBzdGF0aXNjYWxseSBjb3JyZWxhdGVkLgoKCmBgYHtyfQoKbGlicmFyeSgiZ2dwdWJyIikKCmRmX3Jldl9zcGVuZAoKZ2dzY2F0dGVyKGRmX3Jldl9zcGVuZCwgeCA9ICJzcGVuZCIsIHkgPSAidG90YWxfcmV2ZW51ZSIsIAogICAgICAgICAgYWRkID0gInJlZy5saW5lIiwgY29uZi5pbnQgPSBUUlVFLCAKICAgICAgICAgIGNvci5jb2VmID0gVFJVRSwgY29yLm1ldGhvZCA9ICJwZWFyc29uIiwKICAgICAgICAgIHhsYWIgPSAiVG90YWwgQWRTcGVuZCIsIHlsYWIgPSAiVG90YWwgUmV2ZW51ZSAtIDMgc291cmNlcyAiKQoKYGBgCklzIHRoZSByZWxhdGlvbiBsaW5lYXI/IFllcywgZm9ybSB0aGUgcGxvdCBhYm92ZSwgdGhlIHJlbGF0aW9uc2hpcCBpcyBsaW5lYXIuCgpub3cgd2UganN1dCBuZWVkIHRvIHZlcmlmeSB0aGF0IHRoZSBkYXRhIGZyb20gZWFjaCBvZiB0aGUgMiB2YXJpYWJsZXMgKHJldmVudWUsIHNwZW5kKSBmb2xsb3cgYSBub3JtYWwgZGlzdHJpYnV0aW9uLgoKYGBge3J9CgpkZl9yZXZfc3BlbmQKCgojIFNoYXBpcm8tV2lsayBub3JtYWxpdHkgdGVzdCBmb3IgcmV2ZW51ZQpzaGFwaXJvLnRlc3QoZGZfcmV2X3NwZW5kJHRvdGFsX3JldmVudWUpICMgPT4gcC12YWx1ZSA8IDIuMmUtMTYKCiMgU2hhcGlyby1XaWxrIG5vcm1hbGl0eSB0ZXN0IGZvciBzcGVuZApzaGFwaXJvLnRlc3QoZGZfcmV2X3NwZW5kJHNwZW5kKSAjID0+IHAtdmFsdWUgPCAyLjJlLTE2CgoKIyByZXZlbnVlIApnZ3FxcGxvdChkZl9yZXZfc3BlbmQkdG90YWxfcmV2ZW51ZSwgeWxhYiA9ICJyZXZlbnVlIikKIyBzcGVuZApnZ3FxcGxvdChkZl9yZXZfc3BlbmQkc3BlbmQsIHlsYWIgPSAic3BlbmQiKQpgYGAKClNpbmNlIHRoZSBwLXZhbHVlIGlzIHByYWN0aWNhbGx5IHplcm8sIHdlIGNhbiByZWplY3QgdGhlIG51bGwgaHlwb3RoZXNpcyB0aGF0IGl0IGlzIG5vcm1hbCBkaXN0cmlidXRlZCBib3RoIHZhcmlhYmxlcwoKc28gd2UgY2FuIHVzZSBhbm90aGVyIHRlc3QgdG8gY2hlY2sgYXNvY2lhdGlvbgpLZW5kYWxsIHJhbmsgY29ycmVsYXRpb24gdGVzdCAtIEtlbmRhbGzigJlzIHRhdSAKClRoaXMgaXMgdXNlZCB0byBlc3RpbWF0ZSBhIHJhbmstYmFzZWQgbWVhc3VyZSBvZiBhc3NvY2lhdGlvbi4gVGhpcyB0ZXN0IG1heSBiZSB1c2VkIGlmIHRoZSBkYXRhIGRvIG5vdCBuZWNlc3NhcmlseSBjb21lIGZyb20gYSBiaXZhcmlhdGUgbm9ybWFsIGRpc3RyaWJ1dGlvbi4KClRoZSBjb3JyZWxhdGlvbiBjb2VmZmljaWVudCBiZXR3ZWVuIHJldmVudWUgYW5kIHNwZW5kIGFyZSAwLjcyMDkyNDMgYW5kIHRoZSBwLXZhbHVlIDwgMi4yZS0xNi4KRXhpc3QgY29ycmVsYXRpb24KCmJ1dCB3ZSBrbm93IGl0IGlzIG5vdCBjYXVzYXRpb24KCmBgYHtyfQpyZXMyIDwtIGNvci50ZXN0KGRmX3Jldl9zcGVuZCR0b3RhbF9yZXZlbnVlLCBkZl9yZXZfc3BlbmQkc3BlbmQsICBtZXRob2Q9ImtlbmRhbGwiKQpyZXMyCmBgYAoKCldlIGtub3cgdGhhdCB0aGV5IGFyZSBoaWdobHkgY29ycmVsYXRlZC4gV2UgY2FuIHVzZSB0aW1lIHNlcmllcyB0byBzZWUgdGhlIG1hZ25pdHVkZSBvZiB0aGUgZWZmZWN0IG9mIG1hcmtldGluZyB1c2luZyBpdCBhcyByZWdyZXNzb3IKClZpc3VhbGx5IGluIHRoZSBwcmV2aW91cyBwbG90IG9mIHJldmVudWUsIHdlIGNhbiBzZWUgaXQgaXMgc3RhdGlvbmFyeSwgYW5kIHRoZXJlIGlzIHNlYXNvbmFsaXR5LiBXZSBjYW4gdXNlIGF1dG8uYXJpbWEgdG8gY2FsY3VsYXRlIHRoZSByZWdyZXNzb3IgZmFjdG9yCgpUaGlzIGlzIGEgdmVyeSBiYXNpYyBhbmFseXNpcywgYnV0IGl0IGhlbHBzIHRvIHVuZGVyc3RhbmQgaW4gb3ZlcmFsbCBob3cgdGhlIG1hcmtldGluZyBlZmZvcnRzIGFyZSBjcmVhdGluZyBhbiBlZmZlY3QgaW4gdGhlIHJldmVudWUuCgoxLjQ4OTA3MSBpcyB0aGUgcmVncmVzc29yIGZhY3RvcgoKIyMjIyBJbnRlcnByZXRhdGlvbiBvZiBzYWxlcyBpbmNyZWFzZTogRm9yIGV2ZXJ5ICQxLDAwMCAodGhlIHVuaXQgb2YgYWQgc3BlbmQgaXMgaW4gdGhvdXNhbmRzIFVTRCkgaW5jcmVhc2UgaW4gYWR2ZXJ0aXNpbmcgc3BlbmQsIHRoZSB1bml0IHNhbGVzIGluY3JlYXNlIGJ5IDEuNDg5MSAtIEl0IGlzIGEgcG9zaXRpdmUgZWZmZWN0LCBzbyBBQ01FIHByb2JhYmx5IGlzIHVzaW5nIFJFQ0FTVCBhbmQgdGhlIG1hcmtldGluZyBjYW1wYWlnbiBpcyBwZXJmb3JtYWluZyB3ZWxsCgoKCmBgYHtyfQojIHdlIG5lZWQgdG8gY29udmVydCB0aGUgZGFpbHkgZGF0YSB0byB3ZWVrbHkKd2Vla25vIDwtIGFzLm51bWVyaWMoZGZfcmV2X3NwZW5kJGRhdGUgLSBkZl9yZXZfc3BlbmQkZGF0ZVsxXSkgJS8lIDcKV2VlayA8LSBkZl9yZXZfc3BlbmQkZGF0ZVsxXSArIDcgKiB3ZWVrbm8KZGZfYnlfd2VlayA8LSBhZ2dyZWdhdGUoY2JpbmQodG90YWxfcmV2ZW51ZSxzcGVuZCkgIH4gV2VlaywgZGZfcmV2X3NwZW5kLCBzdW0pCgojdG8gY2hlY2sgaXQgbG9va3MgZ29vZApkZl9ieV93ZWVrCgojQ29udmVydCBkYXRhZnJhbWUgdG8gdGltZSBzZXJpZXMgb2JqZWN0IHVzaW5nIHRoZSB0cygpIGZ1bmN0aW9uCmFjbWVfdHNfd2VlayA8LSB0cyhkYXRhID0gZGZfYnlfd2Vla1ssYygyLDMpXSkKIyBUaW1lIHBsb3Qgb2YgYm90aCB2YXJpYWJsZXMKYXV0b3Bsb3QoYWNtZV90c193ZWVrLCBmYWNldHMgPSBUUlVFKQoKIyBGaXQgQVJJTUEgbW9kZWwKZml0IDwtIGF1dG8uYXJpbWEoYWNtZV90c193ZWVrWywgInRvdGFsX3JldmVudWUiXSwgeHJlZyA9IGFjbWVfdHNfd2Vla1ssICJzcGVuZCJdLCAKICAgICAgICAgICAgICAgICAgc3RhdGlvbmFyeSA9IFRSVUUsIHNlYXNvbmFsPVRSVUUsIAogICAgICAgICAgICAgICAgICBhbGxvd2RyaWZ0ID0gRkFMU0UsIGFsbG93bWVhbiA9IEZBTFNFICkjLGxhbWJkYSA9IExhbWJkYSkKCmZpdAoKI1RoZSBhdXRvLmFyaW1hIGZ1bmN0aW9uIGhhcyBmaXQgYSBsaW5lYXIgcmVncmVzc2lvbiBtb2RlbCB0byB0aGUgYWR2ZXJ0aXNpbmcgdmFyaWFibGUgYW5kIGFuIGFuIEFSSU1BIG1vZGVsIHRvIHRoZSB0aW1lIHNlcmllcyAoZGF0ZSkgdmFyaWFibGUuIEFSSU1BKDIsMCwxKQoKI1doYXQgaXMgdGhlIGluY3JlYXNlIGluIHNhbGVzIGZvciBlYWNoIHVuaXQgaW5jcmVhc2UgaW4gYWR2ZXJ0aXNpbmc/CnNhbGVzX2luY3JlYXNlIDwtIGNvZWZmaWNpZW50cyhmaXQpWzRdCnNhbGVzX2luY3JlYXNlCgoKCgojQ2hlY2sgUmVzaWR1YWxzIHRvIHZlcmlmeSB0aGF0IHRoZSByZXNpZHVhbHMgYXJlIHdoaXRlIG5vaXNlIGFuZCBub3JtYWwgZGlzdHJpYnV0ZWQgLSBJbmRlZWQgdGhleSBhcmUgc28gdGhlIG1vZGVsIGlzIGFjY2VwdGFibGUKY2hlY2tyZXNpZHVhbHMoZml0KQoKYGBgCgoKCg==
Comment: https://getrecast.com/modern-media-mix-modeling/.